Algorithm Instance Footprint: Separating Easily Solvable and Challenging Problem Instances

In black-box optimization, it is essential to understand why an algorithm instance works on a set of problem instances while failing on others and provide explanations of its behavior. We propose a methodology for formulating an algorithm instance footprint that consists of a set of problem instances that are easy to be solved and a set of problem instances that are difficult to be solved, for an algorithm instance. This behavior of the algorithm instance is further linked to the landscape properties of the problem instances to provide explanations of which properties make some problem instances easy or challenging. The proposed methodology uses meta-representations that embed the landscape properties of the problem instances and the performance of the algorithm into the same vector space. These meta-representations are obtained by training a supervised machine learning regression model for algorithm performance prediction and applying model explainability techniques to assess the importance of the landscape features to the performance predictions. Next, deterministic clustering of the meta-representations demonstrates that using them captures algorithm performance across the space and detects regions of poor and good algorithm performance, together with an explanation of which landscape properties are leading to it.Comment: To appear at GECCO 202

Assessing the Generalizability of a Performance Predictive Model

A key component of automated algorithm selection and configuration, which in most cases are performed using supervised machine learning (ML) methods is a good-performing predictive model. The predictive model uses the feature representation of a set of problem instances as input data and predicts the algorithm performance achieved on them. Common machine learning models struggle to make predictions for instances with feature representations not covered by the training data, resulting in poor generalization to unseen problems. In this study, we propose a workflow to estimate the generalizability of a predictive model for algorithm performance, trained on one benchmark suite to another. The workflow has been tested by training predictive models across benchmark suites and the results show that generalizability patterns in the landscape feature space are reflected in the performance space.Comment: To appear at GECCO 202

The organisation of physiotherapy for people with multiple sclerosis across Europe: a multicentre questionnaire survey

Background Understanding the organisational set-up of physiotherapy services across different countries is increasingly important as clinicians around the world use evidence to improve their practice. This also has to be taken into consideration when multi-centre international clinical trials are conducted. This survey aimed to systematically describe organisational aspects of physiotherapy services for people with multiple sclerosis (MS) across Europe. Methods Representatives from 72 rehabilitation facilities within 23 European countries completed an online web-based questionnaire survey between 2013 and 2014. Countries were categorised according to four European regions (defined by United Nations Statistics). Similarities and differences between regions were examined. Results Most participating centres specialized in rehabilitation (82 %) and neurology (60 %), with only 38 % specialising in MS. Of these, the Western based Specialist MS centres were predominately based on outpatient services (median MS inpatient ratio 0.14), whilst the Eastern based European services were mostly inpatient in nature (median MS inpatient ratio 0.5). In almost all participating countries, medical doctors - specialists in neurology (60 %) and in rehabilitation (64 %) - were responsible for referral to/prescription of physiotherapy. The most frequent reason for referral to/prescription of physiotherapy was the worsening of symptoms (78 % of centres). Physiotherapists were the most common members of the rehabilitation team; comprising 49 % of the team in Eastern countries compared to approximately 30 % in the rest of Europe. Teamwork was commonly adopted; 86 % of centres based in Western countries utilised the interdisciplinary model, whilst the multidisciplinary model was utilised in Eastern based countries (p = 0.046). Conclusion This survey is the first to provide data about organisational aspects of physiotherapy for people with MS across Europe. Overall, care in key organisational aspects of service provision is broadly similar across regions, although some variations, for example the models of teamwork utilised, are apparent. Organisational framework specifics should be considered anytime a multi-centre study is conducted and results from such studies are applied.PubMedWoSScopu

Improving Nevergrad’s Algorithm Selection Wizard NGOpt Through Automated Algorithm Configuration

International audienceAlgorithm selection wizards are effective and versatile tools that automatically select an optimization algorithm given high-level information about the problem and available computational resources, such as number and type of decision variables, maximal number of evaluations, possibility to parallelize evaluations, etc. State-of-the-art algorithm selection wizards are complex and difficult to improve. We propose in this work the use of automated configuration methods for improving their performance by finding better configurations of the algorithms that compose them. In particular, we use elitist iterated racing (irace) to find CMA configurations for specific artificial benchmarks that replace the hand-crafted CMA configurations currently used in the NGOpt wizard provided by the Nevergrad platform. We discuss in detail the setup of irace for the purpose of generating configurations that work well over the diverse set of problem instances within each benchmark. Our approach improves the performance of the NGOpt wizard, even on benchmark suites that were not part of the tuning by irace